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Abstract 

Using  data  from  the  "Armed  Forces  2002  Sexual  Harassment  Survey,"  this  paper 
analyzed  the  extent  to  which  respondents  refused  to  report  experiences  of  sexual 
harassment  (i.e.,  the  responses  were  missing  for  that  question).  Specifically,  the  total 
percentages  reporting  personally  experiencing  sexual  harassment  were  compared  to  the 
percentage  of  missing  cases  for  that  question.  Data  was  then  aggregated  to  the  unit  level 
and  quartiles  based  on  percentages  reporting  sexist  behavior  in  their  units  were  created. 
Finally,  the  level  of  reported  sexist  behavior  was  compared  to  the  percentages  with 
missing  values  on  the  sexual  harassment  question.  Data  analysis  revealed  that  the  units  in 
the  “worst”  quartile  (i.e.,  highest  reports  of  sexist  behavior)  had  the  most  missing 
responses  on  the  sexual  harassment  question. 


Opinions  expressed  in  this  report  are  those  of  the  author  and  should  not  be  construed  to  represent  the 
official  position  of  DEOMI,  the  U.S.  military  services,  or  the  Department  of  Defense. 


Exploring  Missing  Values  3 


Table  of  Contents 

Abstract  . 2 

Introduction . 4 

Method  . 5 

Sample . 5 

Variable  Construction  and  Survey . 6 

Analysis . 7 

Discussion . 9 

References . 10 

Appendix  A . 13 

Figure  1A. 

Missing  Values  on  Sexual  Harassment  Questions  by  Demographic  Variables . 15 

Figure  2A. 

Personal  Experiences  of  Sexual  Harassment  and  Missing  Values  on  Sexual 

Harassment  Questions  by  Perceptions  of  Sexism  in  a  Unit  (Quartiles) . 16 

Appendix  B  . 17 

Table  IB 

Percentage  of  Reporting  Sexual  Harassment  and  Missing  Cases  Based  on  Quartile 
of  Unit  with  Respect  to  Reported  Sexist  Environment  on  DEOCS  3.3  . 18 


Footnote 


19 


Exploring  Missing  Values  4 


Exploring  Missing  Values  on  Responses  to  Experienced  and  Labeled  Event  as 

Harassment  in  2004  Reserves  Data 

Most  research  devoted  to  survey  methodology  indicates  that  response  rates  have 
been  declining  over  time  -  both  overall  survey  responses  and  responses  to  individual 
questions  (Berk,  Schur,  &  Feldman,  2007;  Hajiabdolbaghi,  et  al.,  2007;  Smyth,  Dillman, 
Christian,  &  Stem,  2006;  Tomaskovik-Devey,  Leiter,  &  Thompson,  1994).  While  there 
are  several  reasons  for  this  occurrence,  it  is  likely  that  military  surveys  are  most  impacted 
by:  a  growing  lack  of  trust  in  government  entities,  overlapping  surveys,  and  over¬ 
surveyed  respondents.  For  example,  Tomaskovik-Devey  et  al.  (1994)  found  that  the  most 
common  reason  individuals  reported  for  not  responding  to  organizational  surveys  was 
that  they  did  not  want  to  divulge  confidential  information  and  this  was  further  impacted 
by  the  idea  that  the  responses  were  forwarded  or  sent  directly  to  headquarters. 

There  is  a  growing  sense  that  military  personnel  are  a  “captive  audience”  and  as  a 
result,  are  asked  to  complete  too  many  (and  often  overlapping)  surveys  (Edwards, 
Rosenfeld,  Booth-Kewley,  &  Thomas,  1996;  Newell,  Rosenfeld,  Harris,  &  Hindelang, 
2004).  This  will  reinforce  disinterest  and  poor  motivation,  which  are  further  compounded 
if  individuals  believe  they  have  a  limited  capacity  to  respond  to  certain  questions. 
Because  the  general  culture  of  the  military  suggests  that  “tattling”  about  negative  events 
is  unacceptable,  too  often  only  those  who  are  not  afraid  of  being  labeled  a 
“troublemaker”  are  likely  to  identify  experiences  of  sexual  harassment  and  assault  and  as 
a  result,  report  them  (even  on  a  confidential  survey)  (see  for  example,  Firestone  &  Harris, 
2003;  Malovich  &  Stake,  1990;  Stockdale  &  Vaux,  1993;  Saal,  1996;  Thomas,  1995).  This 
report  examines  the  cases  with  missing  responses  to  the  sexual  harassment  questions  in  the 
“2004  Workplace  and  Gender  Relations  Survey  of  Reserve  Component  Members  ” 
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(WGRR)  to  test  whether  or  not  there  appears  to  be  a  pattern  indicating  serious  bias  in 
responses. 

Method 

Our  research  examined  a  sample  of  respondents  from  the  “Armed  Forces  2002 
Sexual  Harassment  Survey,”  (Lipari  &  Lancaster,  2003)  conducted  for  the  Office  of  the 
Secretary  of  Defense  by  the  Defense  Manpower  Data  Center.  This  was  a  "worldwide 
scientific  survey  of  how  men  and  women  work  together  in  the  ...  Active-duty  Military 
Services  ...”  The  stated  purpose  of  the  survey  was  "[t]o  assess  the  prevalence  of  sexual 
harassment  and  other  unprofessional,  gender-related  behaviors...."  (Lipari  &  Lancaster, 
2003,  p.  6).  The  instrument  “was  based  on  the  1995  Form  B  questionnaire  and 
incorporated  further  psychometric  and  theoretical  advances  in  sexual  harassment 
research”  (Lipari  &  Lancaster,  2003,  p.  6).  Specifically,  we  compared  the  total 
percentages  reporting  personally  experiencing  sexual  harassment  to  the  percentage  of 
missing  cases  for  that  question.  We  then  aggregated  the  data  to  unit  level  and  created 
quartiles  based  on  percentages  reporting  sexist  behavior  in  their  units.  Finally,  we 
examined  the  level  of  reported  sexist  behavior  compared  to  the  percentages  with  missing 
values  on  the  sexual  harassment  question. 

Sample 

A  single-stage,  stratified  random  sample  of  60,415  respondents  was  drawn  for  the 
survey,  representing  male  and  female  enlisted  personnel  and  officers  in  the  Army,  Navy, 
Marines,  Air  Force,  and  Coast  Guard.  Data  were  collected  by  mail  and  via  the  Web,  with 
one-third  of  respondents  returning  responses  via  the  internet.  A  total  of  19,960  usable 
surveys  were  returned  for  a  response  rate  of  36%  (see,  Flores-Cervantes,  Valiant, 
Harding,  &  Bell,  2003).  The  original  sample  includes  10,235  males  and  9,725  females, 
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illustrating  the  oversampling  of  women.  The  sampling  frame  was  stratified  by  service 
branch,  sex,  pay  grade,  race/ethnicity,  likelihood  of  deployment,  and  geographic  location 
(Elig,  2003).  A  series  of  weighting  schemes  was  developed  by  the  original  survey  team  at 
the  Defense  Manpower  Data  Center  tied  to  branch  of  service,  rank,  sex,  and  race  and  to 
test  for  non-response  bias.  The  full  weights  provide  estimated  numbers  of  respondents 
that  approximate  the  total  active  force  as  of  December  2001  (Lipari  &  Lancaster,  2003,  p. 
5).  To  illustrate  the  impact  of  the  weighting,  there  are  16,154  weighted  male  respondents 
(84.8%)  and  2,906  weighted  female  respondents  (15.2%),  for  a  total  of  19,060  weighted 
cases. 

Variable  Construction  and  Survey 

Among  the  items  in  the  “Gender  Related  Experiences  in  the  Military  in  the  Past 
12  Months”  section  of  the  survey,  respondents  were  asked  the  following: 

In  this  question  you  are  asked  about  sex/gender  related  talk  and/or  behavior  that 

was  unwanted,  uninvited,  and  in  which  you  did  not  participate  willingly. 

How  often  during  the  past  12  months  have  you  been  in  situations  involving 

>  Military  Personnel 

•  On-or  off-duty 

•  On-or  off  installations  or  ship;  and/or 

>  Civilian  Employees  and/or  Contractors 

•  In  your  workplace  or  on  your  installation/ship 

Where  one  or  more  of  these  individuals  (of  either  gender). . . 

Respondents  were  then  provided  a  list  of  19  items  and  asked  whether  that  item 
had  occurred  “very  often,”  “often,”  “sometimes,”  “once  or  twice,”  or  “never.”  We 
recoded  the  first  four  responses  in  an  “ever”  occurred  category  with  a  value  of  1 ;  “never” 
was  coded  0.  Based  on  the  original  statements,  we  identified  individualistic  forms  of 
sexual  harassment  that  are  personal  and  frequently  directly  physical  in  nature,  and  leave 
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little  room  for  misinterpretation  by  either  the  victim  or  the  perpetrator  (e.g.,  sexual 
assault,  touching,  sexual  phone  calls).  This  form  can  be  differentiated  from  a  broader 
category  of  more  public,  environmental  harassment  (e.g.,  jokes,  whistles,  suggestive 
looks).  The  latter  actions  can  be  experienced  even  if  directed  at  another  individual,  and 
are  ambiguous  enough  to  leave  their  interpretation  dependent  on  the  environmental 
context.1  Respondents  were  initially  classified  as  having  experienced  individualistic  or 
environmental  unwanted,  uninvited  sexual  behavior,  or  any  form  (individualistic, 
environmental,  or  both).  We  focus  on  the  separate  categories  of  environmental  and 
individual  harassment  for  this  research. 

Respondents  were  then  asked  whether  they  considered  “ANY  of  the 
behaviors. .  .which  YOU  MARKED  AS  HAPPENING  TO  YOU  ...  to  have  been  sexual 
harassment  [emphases  part  of  original  survey]. ’’Responses  included  “none  were  sexual 
harassment,”  “some  were  sexual  harassment,”  “some  were  not  sexual  harassment,”  and 
“all  were  sexual  harassment.”  This  variable  was  dichotomized  to  indicate  whether  any 
events  were  labeled  as  sexual  harassment,  or  none  were  labeled  as  harassment.  Another 
question  asked  “Did  you  report  this  situation  to  any  of  the  following 
installation/Service/DoD  individuals  or  organizations.”  The  responses  included 
references  to  the  various  official  channels  for  reporting.  Individuals  who  responded  “yes” 
to  any  of  the  categories  were  classified  as  having  used  official  channels  to  report  the 
incident.  Independent  variables  studied  include:  sex  of  respondent,  rank  (i.e.,  junior 
enlisted,  senior  enlisted,  junior  officer,  senior  officer),  marital  status,  and  service  branch. 
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Analysis 

Figure  1A  shows  the  differences  across  various  demographic  indicators  in  relation 
to  the  total  percentage  of  responses  to  the  sexual  harassment  questions  compared  to  the 
percentage  of  missing  responses  to  the  sexual  harassment  questions.  Results  indicate  that 
individuals  with  missing  values  on  the  sexual  harassment  portion  of  the  survey  are  likely 
to  have  one  or  more  of  the  following  characteristics:  Hispanic,  Black,  junior  enlisted, 
female,  and  have  not  completed  college.  All  of  these  categories  are  those  with  less  power 
within  the  military  and  thus  members  of  these  groups  may  be  fearful  of  responding  to  the 
survey  with  honest  answers.  While  in  most  cases  the  percentage  differences  are  small, 
they  do  indicate  the  likelihood  that  incidences  of  sexual  harassment  are  under  reported  on 
these  surveys.  This  is  further  emphasized  by  our  analysis  of  missing  values  on  the  sexual 
harassment  question  on  the  DEOCS  3.3  at  the  unit  level. 

Table  IB  displays  the  percentage  of  individuals  stating  they  personally 
experienced  sexual  harassment  and  the  percentage  of  missing  cases  classified  by  quartiles 
(perception  of  sexism  in  a  unit).  The  “worst”  category  indicates  units  with  the  highest 
levels  of  reported  sexist  behavior,  and  the  “best”  category  indicates  the  lowest  levels  of 
reported  sexist  behavior.  It  is  very  clear  that  the  units  in  the  “worst”  category  had  the 
most  missing  cases  on  sexual  harassment.  Almost  half  of  all  missing  values  cases  are 
from  those  units  in  the  worst  quartile. 

This  is  even  more  pronounced  in  Figure  2A.  The  blue  columns  represent  the 
percent  of  missing  values  on  the  sexual  harassment  question  located  in  the  quartiles.  It  is 
very  clear  looking  at  the  graph  that  the  largest  percentage  of  missing  values  is  found  in 
the  units  that  were  classified  as  “worst”  with  respect  to  reported  sexist  behaviors.  While 
we  cannot  directly  test  our  statement,  this  is  highly  suggestive  that  in  units  where  sexist 
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behaviors  are  more  prevalent,  individuals  may  be  afraid  to  provide  honest  answers  about 
sexual  harassing  behaviors  and  as  a  result  skip  the  sexual  harassment  questions. 

Discussion 

Our  results  suggest  that  individuals  who  feel  powerless  or  who  work  in  a  sexist 
environment,  may  be  afraid  to  report  sexual  harassment  (even  on  confidential  surveys). 
Reults  suggest  that  the  surveys  likely  under  report  actual  sexual  harassment  experiences. 
Furthermore,  the  aggregate  data  strongly  support  our  earlier  work  (Firestone  &  Harris, 
1994;  1997;  1999;  2003;  2007;  Harris  &  Firestone,  1997)  indicating  that  environmental 
factors  are  key  in  preventing  and/or  controlling  sexual  harassment. 
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Figure  Caption 


Figure  1A.  Missing  values  on  sexual  harassment  questions  by  demographic  variables. 


Figure  2A.  Personal  experiences  of  sexual  harassment  and  missing  values  on  sexual 
harassment  questions  by  perceptions  of  sexism  in  a  unit  (quartiles). 
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Table  IB 

Percentage  Reporting  Sexual  Harassment  and  Missing  Case  Based  on  Quartile  of  Unit 
with  Respect  to  Reported  Sexist  Environment  on  DEOCS  3.3 


Worst 

Second 

Third 

Best 

Total 

Yes,  Harassed 

42 

24.4 

20.6 

13 

100 

No 

22.9 

25.3 

25.5 

26.3 

100 

Missing 

49.5 

29.3 

17.3 

12.9 

100 
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Foot  Note 

Because  the  questions  used  in  the  2002  survey  were  not  an  exact  match  to  the  questions 
from  the  original  1988  survey,  our  conceptualizations  for  individual  and  environmental 
harassment  are  a  broad  match,  but  not  an  exact  match  of  our  earlier  research.  For  a 
description  of  the  statements  classified  as  individual  or  environmental  harassment,  see 
Firestone  and  Harris,  1994. 


